Use arrow kernels for bitwise operations #6098
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alamb
commented
Apr 23, 2023
| let right = | ||
| Arc::new(Int32Array::from(vec![Some(1), Some(3), Some(7)])) as ArrayRef; | ||
| let mut result = bitwise_and(left.clone(), right.clone())?; | ||
| let mut result = bitwise_and_dyn(left.clone(), right.clone())?; |
There was a problem hiding this comment.
renamed these kernels to match the naming scheme in arrow-rs (where _dyn is used to refer to kernels that take ArrayRef
| create_dyn_kernel!(bitwise_xor_dyn, bitwise_xor); | ||
| create_dyn_kernel!(bitwise_and_dyn, bitwise_and); | ||
|
|
||
| pub(crate) fn bitwise_shift_right_dyn( |
There was a problem hiding this comment.
Sadly there are no shift_right or shift_left in arrows
There was a problem hiding this comment.
I also found the upstream tracking ticket in arrow -- apache/arrow-rs#2741 and added links in the code here
viirya
reviewed
Apr 23, 2023
viirya
reviewed
Apr 23, 2023
viirya
approved these changes
Apr 23, 2023
Co-authored-by: Liang-Chi Hsieh <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Related to #6066
Rationale for this change
While reviewing #6093 from @RTEnzyme I noticed there were kernels in arrow-rs for some of the bitwise operations: https://docs.rs/arrow/latest/arrow/compute/kernels/bitwise/index.html
What changes are included in this PR?
Use the kernels from arrow-rs rather than redefining them in datafusion
Are these changes tested?
Covered by existing tests
Are there any user-facing changes?
No